Improving the Output from Software that Generates Multiple Choice Question (MCQ) Test Items Automatically using Controlled Rhetorical Structure Theory

A combination of established theories [1].[2].[3].[4] are applied in an attempt to improve the output from a system [5],[6] which automatically generates MCQ (Multiple Choice Question) test items from source documents. The literature observes that NLG (Natural Language Generation) system evaluation is non-trivial [7] and so the method is evaluated using a process suited to the featured domain [8]. The experiment intersperses 38 MCQ test items whose question stems have been generated using Controlled Rhetorical Structure Theory (CRST) with 62 manually created MCQ test items to form an item bank. A usability score is assigned to each item by a domain expert and these scores are used in the evaluation of the effectiveness of the method. The results provide some evidence to support the incorporation of CRST into future versions of the software. .